Investigating the internal structure of multiple mini interviews-A perspective from Pakistan.

BACKGROUND: Healthcare professionals require many personal attributes in addition to cognitive abilities and psychomotor skills for competent practice. Multiple Mini- Interviews are being employed globally to assess personality attributes of candidates for selection in health professions education at all level of entry; these attributes are namely, communication skills, critical thinking, honesty, responsibility, health advocacy, empathy and sanctity of life. Considering the high stakes involved for students, faculty, institutions and the society, rigorous quality assurance mechanisms similar to those used for student assessment must be employed for student selection, throughout the continuum of medical education. It is a difficult undertaking as these psychological constructs are difficult to define and measure. Though considered to yield reliable and valid scores, studies providing multiple evidences of internal structure especially dimensionality of Multiple Mini-Interviews are sparse giving rise to questions if they are measuring a single or multiple constructs and even if they are measuring what they are purported to be measuring. OBJECTIVE: The main objective is to provide statistical support of the multi-dimensional nature of our Multiple Mini Interviews, hypothesized a-priori, through CFA. Another objective is to provide multiple evidences for the internal structure. Our study highlights the link between content and internal structure evidences of the constructs, thus establishing that our Multiple Mini Interviews measure what they were intended to measure. METHOD: After securing permission from the Institutional review board, an a-priori seven factor-model was hypothesized based on the attributes considered most essential for the graduating student of the institution. After operationally defining the attributes through extensive literature search, scenarios were constructed to assess them. A 5-point rating scale was used to rate each item on the station. A total 259 students participated in the multiple mini interviews over a period of three days. A training workshop had been arranged for the participating faculty. RESULTS: The reliability coefficient using Cronbach's alpha were calculated (range from 0.73 to 0.94), Standard Error of Measurement (ranged from 0.80 to1.64), and item to station-total correlation ranged from 0.43-0.50 to 0.75-0.83. Inter-station correlation was also determined. Confirmatory factor analysis endorsed the results of Exploratory factor analysis in the study revealing a seven model fit with multiple indices of Goodness-of-fit statistics such as Root mean square error of approximation (RMSEA) value 0.05, Standardized root mean square residual (SRMR) value with less than 0.08. All these indices showed that model fit is good. The Confirmatory factor analysis confirmed the multi-dimensional nature of our MMIs and also confirmed that our stations measured the attributes that they were supposed to measure. CONCLUSION: This study adds to the validity evidence of Multiple Mini-Interviews, in selection of candidates, with required personality traits for healthcare profession. It provides the evidence for the multi-dimensional structure of Multiple Mini interviews administered with multiple evidences for its internal structure and demonstrates the independence of different constructs being measured.

Preference Signaling and the Integrated Plastic Surgery Match: A National Survey Study.

OBJECTIVE: Rooted in economics market strategies, preference signaling was introduced to the Plastic Surgery Common Application (PSCA) in 2022 for integrated plastic surgery residency applicants. This study surveyed program and applicant experience with preference signaling and assessed how preference signals influenced likelihood of interview invitations. DESIGN: Two online surveys were designed and distributed to all program directors and 2022-2023 applicants to integrated plastic surgery. Opinions regarding the utility of preference signaling were solicited, and the influence of preference signals on likelihood of interview offers was assessed. SETTING: All integrated plastic surgery programs. PARTICIPANTS: All 88 program directors and 2022-2023 applicants to integrated plastic surgery. RESULTS: A total of 45 programs and 99 applicants completed the survey (response rates, 54.2% and 34.2%, respectively). Overall, 79.6% of applicants and 68.9% of programs reported that preference signals were a useful addition to the application cycle. Programs reported that 41.4% of students who sent preference signals received interview offers, compared to 84.6% of home students, 64.8% of away rotators, and 7.1% of other applicants; overall, students who signaled were 5.8 times more likely to receive an interview offer compared to students who were not home students and did not rotate or signal. After multivariable adjustment, programs with higher Doximity rankings, numbers of away rotators, and numbers of integrated residents per year received more preference signals (all p < 0.05). CONCLUSIONS: Applicants and programs report that preference signaling was a useful addition to the integrated plastic surgery application cycle. Sending preference signals resulted in a higher likelihood of interview offers among nonrotators. Preference signaling may be a useful tool to reduce congestion in the integrated plastic surgery application cycle.

The impact of the medical school admissions interview: a systematic review.

Background: Interviews are considered an important part of the medical school admissions process but have been critiqued based on bias and reliability concerns since the 1950s. To determine the impact of the interview, this systematic review investigated the characteristics and outcomes of medical students admitted with and without interviews. Methods: We searched four literature databases from inception through August 2022; all studies comparing medical students admitted with and without interviews were included. We excluded studies from outside the medical school setting and non-research reports. We reviewed interview type, study design, quality, and outcomes. Results: Eight studies from five institutions across five countries were included. Six reported no demographic differences between students admitted with and without interviews; one found that more men were admitted without than with semi-structured interviews, and both cohorts had similar academic and clinical performance. Structured interviews admitted students who scored higher on clinical exams and social competence and lower on academic exams. Cohorts admitted with and without structured interviews had similar mental health issues by their final year of medical school. Discussion: This review suggests that students admitted with and without unstructured and semi-structured interviews were similar demographically, academically, and clinically. Moreover, structured interviews selected more socially competent students who performed better clinically but worse academically. Further research is needed to determine the impact of the selection interview in medical school admissions.

Contexte: Les entrevues sont considérées comme une composante importante du processus d'admission dans les facultés de médecine, mais elles ont été critiquées depuis les années 1950 sur la base de préoccupations liées à la partialité et à la fiabilité. Afin de déterminer l'impact de l'entrevue, nous avons étudié dans cette revue systématique les caractéristiques et les résultats des étudiants en médecine admis ayant passé ou non une entrevue. Méthodes: Nous avons effectué des recherches dans quatre bases de données bibliographiques depuis leur création jusqu'à août 2022; toutes les études comparant les étudiants en médecine admis avec ou sans entrevue ont été incluses. Nous avons exclu les études réalisées en dehors du cadre des facultés de médecine et les rapports ne relevant pas de la recherche. Nous avons examiné le type d'entrevue, la conception de l'étude, la qualité et les résultats. Résultats: Huit études provenant de cinq établissements dans cinq pays ont été incluses. Six d'entre elles ne font état d'aucune différence démographique entre les étudiants admis avec ou sans entrevue ; l'une d'entre elles a révélé que davantage d'hommes étaient admis sans entrevue qu'avec une entrevue semi-structurée, et que les deux cohortes présentaient des rendements universitaires et cliniques similaires. Les entrevues structurées ont permis d'admettre des étudiants qui ont obtenu de meilleurs résultats aux examens cliniques et compétence sociale et de moins bons résultats aux examens universitaires. Les cohortes admises avec et sans entrevues structurées présentaient des problèmes de santé mentale similaires lors de leur dernière année d'études de médecine. Discussion: Cette étude suggère que les étudiants admis avec et sans entrevues non structurées et semi-structurées étaient similaires d'un point de vue démographique, universitaire et clinique. En outre, les entrevues structurées ont permis de sélectionner des étudiants plus compétents sur le plan social, qui ont obtenu de meilleurs résultats cliniques, mais avec une moins bonne performance sur le plan académique. D'autres recherches sont nécessaires pour déterminer l'impact de l'entrevue de sélection sur les admissions dans les facultés de médecine.

How professional and academic pre-qualifications relate to success in medical education: Results of a multicentre study in Germany.

OBJECTIVE: Every year, many applicants want to study medicine. Appropriate selection procedures are needed to identify suitable candidates for the demanding curriculum. Although research on medical school admissions has shown good predictive validity for cognitive selection methods (undergraduate GPA, aptitude tests), the literature on applicants with professional and/or academic experience prior to entering medical school remains slim. In our study, we therefore aimed to examine the association between academic success in medical school and having previously completed vocational training in the medical field, voluntary service (≥11 months) or an academic degree. METHODS: Data were collected in a multicentre, cross-sectional study at five medical schools in Germany (Baden-Wuerttemberg) from students during medical school (i.e. 3rd-, 6th-, and 10th-semester and final-year students). Academic success was assessed according to scores on the first and second state examinations, the total number of examinations repeated and the number of semesters beyond the standard period of study. For the analysis we calculated ordinal logistic regression models for each outcome variable of academic success. RESULTS: A total of N = 2,370 participants (response rate: RR = 47%) participated in the study. Having completed vocational training was associated with a higher amount of repeated examinations (small effect), while having an academic degree was associated with worse scores on the second state examination (medium effect). No significant association emerged between voluntary service and academic success. CONCLUSION: The results indicate that professional and academic pre-qualifications pose no advantage for academic success. Possible associations with the financing of study and living conditions of students with pre-qualifications were analysed and discussed in an exploratory manner. However, the operationalisation of academic success from objective and cognitive data should be critically discussed, as the benefits of prior experience may be captured by personal qualities rather than examination results.

Mixed Signals: Navigating the Obstetrics and Gynecology Signaling Initiative.

OBJECTIVE: There are few published accounts of the obstetrics and gynecology (OBGYN) specialty-specific experience with a formal signaling program. Prior studies examining other medical specialties' experiences with signaling are quantitative, having not examined the complexity of the residency applicant experience by directly engaging applicants; therefore, this study aimed to describe the lived experiences of OBGYN residency applicants who employed a formal signaling program during the 2022-2023 residency application cycle to assist and guide future residency applicants. DESIGN: A phenomenological approach was chosen to prescribe a common meaning for OBGYN residency applicants' experiences. purposeful sampling was employed to ensure racial, ethnic, and institutional geographic diversity in participant representation. Semi-structured interviews were conducted virtually between April and May 2023. RESULTS: Twenty-five OBGYN residency applicants participated. Fourteen identified as underrepresented in medicine. Four themes emerged: non-uniform decision-making processes, inconsistent guidance, mental health effect, and signaling reflections. Some themes had associated subthemes. Critical aspects of the applicants' journey were revealed, including decision-making dynamics and reliance on trusted advisors. Applicants described tensions and complexities when navigating signaling strategy in relation to abortion education opportunities in a post-Dobbs era. They also conveyed concerns about inconsistencies in signaling guidance, the emotional toll on well-being, and persistent inequities in the application process. They ultimately made recommendations for future directions, including suggestions for more robust advising and improved signaling execution. CONCLUSIONS: This study offers a comprehensive exploration of the experiences of OBGYN residency applicants with formal program signaling. To ensure equity and transparency in the residency application process, it is crucial to not only provide OBGYN residency applicants with clear guidance on signaling, but also encourage a standardized approach for its utilization by residency programs.

The disparate impacts of college admissions policies on Asian American applicants.

There is debate over whether Asian American students face additional barriers, relative to white students, when applying to selective colleges. Here we present the results from analyzing 685,709 applications submitted over five application cycles to 11 highly selective colleges (the "Ivy-11"). We estimate that Asian American applicants had 28% lower odds of ultimately attending an Ivy-11 school than white applicants with similar academic and extracurricular qualifications. The gap was particularly pronounced for students of South Asian descent (49% lower odds). Given the high yield rates and competitive financial aid policies of the schools we consider, the disparity in attendance rates is likely driven, at least in part, by admissions decisions. In particular, we offer evidence that this pattern stems from two factors. First, many selective colleges give preference to the children of alumni in admissions. We find that white applicants were substantially more likely to have such legacy status than Asian applicants. Second, we identify geographic disparities potentially reflective of admissions policies that disadvantage students from certain regions of the United States. We hope these results inform discussions on equity in higher education.

Diversity in osteopathic medical school admissions and the COMPASS program: an update.

In the United States, the 40 colleges of osteopathic medicine and 157 schools of allopathic medicine face challenges in recruiting candidates who are underrepresented in medicine (URiM), and gaps in racial disparity appear to be widening. In this commentary, the authors provide an analysis of the data collected from 8 years of conducting a URiM recruitment and welcoming social events. The event is sponsored by a student special interest group called Creating Osteopathic Minority Physicians Who Achieve Scholastic Success (COMPASS) at the Touro College of Osteopathic Medicine - New York (TouroCOM-NY). The results of the 8-year data analysis supports the conclusion that the COMPASS program has benefited the school environment through increased diversity.

Assessing the predictive validity of pre-admission criteria on dental students' academic performance: a cross-sectional study.

BACKGROUND: This study examines the multifaceted factors influencing academic performance among dental students in Saudi Arabia, exploring the complex relationships between demographic, educational, and psychological variables and students' Grade Point Averages (GPAs) to enhance global dental education admission strategies. METHODS: Data on demographics, academic performance indicators (including GPA, high school grades, test scores), accommodation type, parental education, suspension history, and English proficiency were collected through an English-language electronic questionnaire from 900 dental students across multiple institutions in Riyadh. The study employed Pearson's correlation analysis to examine relationships between GPA and various academic and demographic factors. A detailed regression analysis, using a stepwise selection based on Akaike's information criteria (AIC), identified significant GPA predictors and evaluated the average marginal effects of pre-admission variables on GPA, alongside rigorous diagnostic checks to validate the model's robustness. RESULTS: The study revealed a complex interplay of factors impacting GPA among dental students. High school grades, achievement, and aptitude test scores showed moderate positive correlations with GPA, while negative correlations were observed with age and number of times suspended. Regression analysis highlighted age, marital status, high school grades, and test scores as key predictors, with complex interaction effects demonstrating the layered influences of these factors. For instance, the negative impact of age on GPA was moderated by high school grade and aptitude test score. The analysis also highlighted the significant marginal effects of various pre-admission variables on GPA, such as the negative average impact of age and the positive impact of high school grades. CONCLUSIONS: This study provides valuable insights into the multifaceted determinants of academic success in dental education. Our findings underscore the significant roles of high school grades, achievement, and aptitude test scores, as well as the nuanced influence of age and marital status on GPA. These results advocate for a holistic approach in evaluating candidates for dental programs and have broader implications for global dental education, emphasizing the need for comprehensive admission strategies.

Time to Pivot: A Guide to Holistic Admissions.

ABSTRACT: As an ever-increasing number of physician assistant (PA) programs moves toward holistic admissions, a better understanding of how to achieve their stated admission goals becomes more important. With the June 2023 US Supreme Court decision effectively ending affirmative action in higher education, navigating holistic admissions is now an even greater challenge. In this article, the PA Education Association's Presidents Commission offers a guide for programs to use in implementing holistic admissions at their institutions and key considerations. Is the process mission-driven? Does it follow principles of quality improvement and incorporate ongoing assessment of that process? Using data can be a constructive and insightful way to inform the process. The authors hope that tools, resources, and recommendations offered in this article will serve as valuable resources for any program attempting to institute or improve its holistic admissions process.

Changing Course: Understanding the Legal Landscape of Race-Conscious Admission Practices and Implications to Diversity-Promoting Strategies in Higher Education.

ABSTRACT: The 2023 US Supreme Court holding in Students for Fair Admissions, Inc. v. President and Fellows of Harvard College and Students for Fair Admissions, Inc. v. University of North Carolina, et al. has changed the landscape in higher education admissions processes by declaring affirmative action unlawful. It is imperative that those working in higher education admissions understand the ruling and implement contemporary best practices. Institutions of higher education who wish to promote student diversity are tasked with developing alternative strategies to affirmative action and must steer clear from using race as a determinant factor in admissions decisions. Aiming to identify diversity in individuals' lived experiences, focusing on recruitment efforts that organically lead to diverse applicants, adopting holistic, mission-driven admissions processes, and enhancing robust retention efforts may be effective means of fostering student diversity.

Demographic disparities in dental school selection: An analysis of current UK practices.

INTRODUCTION: Specific social groups remain under-represented within dentistry. While the University Clinical Aptitude Test (UCAT) aims to widen participation in under-represented social groups, there is no evidence in dental education that this aim is being met. MATERIALS AND METHODS: Data over two admission cycles (2012 and 2013), including 3246 applicants to 10 UK dental schools, were analysed. Applicant and selected pools were compared to the UK population. Multiple logistic regression was used to investigate the association between demographic variables and UCAT and receiving an offer of a place at dental school. RESULTS: Over-representation of Female, Asian, least deprived and grammar school groups were found in applicant and selected pools compared with the UK population. White ethnic applicants were significantly more selected than Black (OR 0.25), Asian (OR 0.57) and Mixed (OR 0.80) ethnicities, while least deprived applicants were significantly more selected than most deprived (OR 0.59). Grammar school education increased odds of selection by 1.8 when compared to state school. The addition of UCAT to the model for applicants reduced ethnic disparities but led to disparities between other groups. CONCLUSION: Current widening participation practices focus on attracting applicants from lower socio-economic groups. However, this study showed that ethnicity, sex and educational background biases also affect demographic diversity in dentistry. The UCAT shows promise in levelling the playing field; however, widening access measures will only succeed if selection committees radically change selection processes to address the systemic biases, enabling the dentists of tomorrow to represent the society they serve.

Evaluating a Situational Judgment Test for Use in Medical School Admissions: Two Years of AAMC PREview Exam Administration Data.

PURPOSE: To examine the relationship between the Association of American Medical Colleges (AAMC) Professional Readiness Exam (PREview) scores and other admissions data, group differences in mean PREview scores, and whether adding a new assessment tool affected the volume and composition of applicant pools. METHOD: Data from the 2020 and 2021 PREview exam administrations were analyzed. Two U.S. schools participated in the PREview pilot in 2020 and 6 U.S. schools participated in 2021. PREview scores were paired with data from the American Medical College Application Service (undergraduate grade point averages [GPAs], Medical College Admission Test [MCAT] scores, race, and ethnicity) and participating schools (interview ratings). RESULTS: Data included 19,525 PREview scores from 18,549 unique PREview examinees. Correlations between PREview scores and undergraduate GPAs ( r = .16) and MCAT scores ( r = .29) were small and positive. Correlations between PREview scores and interview ratings were also small and positive, ranging between .09 and .14 after correcting for range restriction. Small group differences in mean PREview scores were observed between White and Black or African American and White and Hispanic, Latino, or of Spanish origin examinees. The addition of the PREview exam did not substantially change the volume or composition of participating schools' applicant pools. CONCLUSIONS: Results suggest the PREview exam measures knowledge of competencies that are distinct from those measured by other measures used in medical school admissions. Observed group differences were smaller than group differences observed with traditional academic assessments and evaluations. The addition of the PREview exam did not substantially change the overall volume of applications or the proportions of out-of-state, underrepresented in medicine, or lower socioeconomic status applicants. While more research is needed, these results suggest the PREview exam may provide unique information to the admissions process without adversely affecting applicant pools.

Medical students' perception of their 'distance travelled' in medical school applications.

INTRODUCTION: Within medical school's holistic review of applicants includes a review of their distance travelled to get to this point in their education. The AAMC defines distance travelled (DT) as, 'any obstacles or hardships you've overcome to get to this point in your education or any life challenges you've faced and conquered'. What medical students consider as their distance travelled has not been explored. The authors sought to identify the factors medical students perceive are important for medical school admissions to consider when assessing someone's 'distance travelled' by asking current medical students to share their DT experiences along with the barriers and facilitators they encountered on their medical school journey. METHODS: The authors conducted semi-structured interviews with US medical students through purposeful sampling methods. The social-ecological model framework was used to develop questions to elicit participants' experiences that contributed to their distance travelled. Interviews were conducted in 2021 and ranged from 60-75 minutes. Transcribed interviews were qualitatively analysed using interpretive description. RESULTS: A total of 31 medical students from seven medical schools were included in the study. Overall, participants defined distance travelled as an applicant's hardships (e.g. being the primary caregiver for a family member) and privileges (e.g. having physician parents) they experienced. Three major themes were identified: (1) individual-level characteristics and factors, (2) interpersonal relationships and (3) aspects of the participants' community and society. DISCUSSION: Our findings show that medical school applicants considered DT to be a valuable component of a holistic medical school admission process. Participants' experiences of DT were varied and complex. Our research suggests that admissions teams for medical schools should incorporate more comprehensive recruitment practices and inclusive methodological frameworks to accurately capture the diversity of identities and experiences of medical school applicants and to consider the factors that shape their journey to medical schools.

A "Red Flag" system adds value to medical school admissions interviews.

INTRODUCTION: Non-cognitive traits should be considered when selecting candidates to study medicine. However, evaluating these traits remains difficult. We explored whether measuring undesirable non-cognitive behaviour ('Red Flags') added value to a medical school admissions system. Red Flags included rudeness, ignoring the contributions of others, disrespectful behaviour, or poor communication. METHODS: Following an admissions interview testing non-cognitive attributes in 648 applicants to a UK medical school, we measured the association between interview score and Red Flag frequency. We tested linear and polynomial regression models to evaluate whether the association was linear or non-linear. RESULTS: In total, 1126 Red Flags were observed. While Red Flags were concentrated among low-scorers, candidates in the highest- and second-highest deciles for interview score still received Red Flags (six and twenty-two, respectively). The polynomial regression model indicated candidates with higher scores received fewer Red Flags, but the association was not linear (F(3644) = 159.8, p = .001, adjusted R2 = 0.42). CONCLUSIONS: The non-linear association between interview score and Red Flag frequency shows some candidates with desirable non-cognitive attributes will still display undesirable-or even exclusionary-non-cognitive attributes. Recording Red Flag behaviour reduces the likelihood such candidates will be offered a place at medical school.

Designing a Situational Judgment Test for Use in Medical School Admissions.

ABSTRACT: It has long been acknowledged that professional competencies are required for success in medical school, residency training, and medical practice. Over the last decade, medical schools have begun to introduce standardized assessments of professional competencies, but many still rely on interviews to assess these competencies, which occur after about half of the applicant pool has already been screened out. In this article, the authors discuss the development, evaluation, and launch of the Association of American Medical Colleges (AAMC) situational judgment test (SJT) for use in medical school admissions. The AAMC SJT is designed to assess an examinee's understanding of effective and ineffective behaviors related to the core competencies for entering medical students, including service orientation, social skills, cultural competence, teamwork, ethical responsibility to self and others, reliability and dependability, resilience and adaptability, and capacity for improvement. The authors evaluate the evidence for the need for SJTs in medical school admissions by exploring common derailers in medical school, gaps in the admissions process regarding information about professional competencies, and the challenge of conducting holistic review in a high-volume context. They summarize existing research from the employment, international medical education, and residency selection contexts suggesting that SJT scores are positively associated with subsequent performance and may add value to the admissions process. The authors discuss 5 goals that were the foundation for developing the AAMC SJT: (1) assess the professional competencies needed for success in medical school using a proven method, (2) enable holistic review in a high-volume admissions context, (3) create and share a program of research to support the appropriate use of SJT scores, (4) signal the need for preparation in professionalism to learners, and (5) balance the need for a new assessment with minimizing the burden and risk for applicants.

Follow the policy: An actor network theory study of widening participation to medicine in two countries.

INTRODUCTION: The slow pace of change in respect of increasing the diversity of medical students suggests powerful actors are reproducing practices to support the status quo. Opening up medicine to embrace diversity thus requires the deconstruction of entrenched processes and practices. The first step in doing so is to understand how the actor-network of widening participation and access to medicine (WP/WA) is constructed. Thus, here we examine how the connections among actors in WP/WA in two different networks are assembled. METHODS: A comparative case study using documents (n = 7) and interviews with staff and students (n = 45) from two medical schools, one United Kingdom and one Australian, was used. We used Callon's moments of translation (problematisation, interessement/operationalisation, enrolment, mobilisation) to map the network of actors as they are assembled in relation to one another. Our main actant was institutional WP to medicine policy (actor-as-policy). RESULTS: Our actor-as-policy introduced five other actors: the medical school, medical profession, high schools, applicants and medical school staff. In terms of problematisation, academic excellence holds firm as the obligatory passage point and focal challenge for all actors in both countries. The networks are operationalised via activities such as outreach and admissions policy (e.g., affirmative action is apparent in Australia but not the UK). High schools play (at best) a passive role, but directed by the policy, the medical schools and applicants work hard to achieve WP/WA to medicine. In both contexts, staff are key mobilisers of WP/WA, but with little guidance in how to enact policy. In Australia, policy drivers plus associated entry structures mean the medical profession exerts significant influence. CONCLUSIONS: Keeping academic excellence as the obligatory passage point to medical school shapes the whole network of WP/WA and perpetuates inequality. Only by addressing this can the network reconfigure.